Skip to content

feat: handle watermark in hash join #7379

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Jan 16, 2023
Merged

feat: handle watermark in hash join #7379

merged 10 commits into from
Jan 16, 2023

Conversation

yuhao-su
Copy link
Contributor

@yuhao-su yuhao-su commented Jan 13, 2023 •
edited
Loading

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.

What's changed and what's your intention?

  • align (take minimal) watermark from left and right
  • state cleaning

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features).
  • All checks passed in ./risedev check (or alias, ./risedev c)

Refer to a related PR or issue link (optional)

close #6268

@github-actions github-actions bot added the type/feature Type: New feature. label Jan 13, 2023
soundOfDestiny
soundOfDestiny approved these changes Jan 13, 2023
View reviewed changes
Copy link
Contributor

@soundOfDestiny soundOfDestiny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@codecov
Copy link

codecov bot commented Jan 16, 2023 •
edited
Loading

Codecov Report

Merging #7379 (1b8fa46) into main (b63df21) will increase coverage by 0.01%.
The diff coverage is 99.20%.

@@            Coverage Diff             @@
##             main    #7379      +/-   ##
==========================================
+ Coverage   73.06%   73.08%   +0.01%     
==========================================
  Files        1068     1068              
  Lines      171706   171815     +109     
==========================================
+ Hits       125464   125577     +113     
+ Misses      46242    46238       -4
Flag Coverage Δ
rust 73.08% <99.20%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/stream/src/executor/hash_join.rs 97.32% <99.09%> (+0.26%) ⬆️
src/stream/src/executor/managed_state/join/mod.rs 92.15% <100.00%> (+0.13%) ⬆️
src/stream/src/executor/mod.rs 52.69% <100.00%> (+0.82%) ⬆️
src/stream/src/executor/watermark/mod.rs 70.90% <100.00%> (+6.62%) ⬆️
src/batch/src/executor/group_top_n.rs 68.42% <0.00%> (-6.44%) ⬇️
src/meta/src/hummock/mock_hummock_meta_client.rs 65.26% <0.00%> (-1.06%) ⬇️
src/storage/src/hummock/compactor/mod.rs 83.48% <0.00%> (-0.16%) ⬇️
src/meta/src/hummock/manager/mod.rs 79.92% <0.00%> (-0.07%) ⬇️
src/stream/src/executor/aggregation/minput.rs 96.49% <0.00%> (+0.10%) ⬆️
src/storage/src/hummock/sstable_store.rs 65.00% <0.00%> (+0.15%) ⬆️
... and 5 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

st1page
st1page approved these changes Jan 16, 2023
View reviewed changes
Copy link
Contributor

@st1page st1page left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mergify mergify bot merged commit ca339df into main Jan 16, 2023
@mergify mergify bot deleted the yuhao/handle_join_watermark branch January 16, 2023 06:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@st1page st1page st1page approved these changes

@soundOfDestiny soundOfDestiny soundOfDestiny approved these changes

@TennyZhuang TennyZhuang Awaiting requested review from TennyZhuang

@fuyufjh fuyufjh Awaiting requested review from fuyufjh

@BugenZhao BugenZhao Awaiting requested review from BugenZhao

Assignees
No one assigned
Labels
type/feature Type: New feature.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Clean state in HashJoinExecutor by watermark
3 participants
@yuhao-su @st1page @soundOfDestiny